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The notice of December 2 1 . 2005 is respectfully traversed. 37 CFR 4 1 .68 (ix) and (x) 
state thai there is to be an appendix wilh any evidence or related decision. The undersigned 
understands this language to mean that, where there is no evidence or related decision, there 
should he no appendix. The requirement to add blank appendices seems strange, annoying, and 
an unlikely interpretation ol' the regulations. Withdrawal of the first notice is accordingly 
respectfully requested. 

Nevertheless, in effort to advance prosecution, a revised appeal brief with blank 
appendices is included. 

Also enclosed is a petition for extension of time. This petition should be considered 
provisional in nature. In other words, if" the PTO determines that docs withdraw the first notice, 
the extension fee should be rc-creditcd to the deposit account. 


Respectfully submitted, 
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TN THE UNITED STATES PATENT AND TRADEMARK OFFICK 
Patent Application Scr. No.: 09/822,1 21 Group Art Unit: 2643 

Filing Date: 3/30/2001 Examiner: WING F. CHAN 

Attorney Docket Number PTI-US 010080 Inventor Name(s): COLMENAREX ET Al ,. 
Confirmation #: 8881 

Title: METHOD AND APPARATUS FOR AUDIO/IMAGE SPEAKER DETECTION AND 
LOCATOR 

Mail Stop Appeal Uriel' 
Commissioner for Patents 
P.O. Box 1450 
Alexandria V A 223 13-1 450 

APPEAL BRIEF fseconri rr.viW) 

Sir: 

This is an appeal from the final rejection ol'Claims 1-25. 

I. REAL PARTY IN INTEREST 

The real party in interest is Kcrainklijkc Philips Electronics, N.V., a corporation of the 
Netherlands. 

II. RELATED APPEALS AND INTERFERENCES 

Applicants arc not aware of any related appeals or interferences. 
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III. STATUS OF CI AIMS 

Claims 1-8 and 10-24 stand rejected under 35 USC 103(a) over WO 99/60788 ("Potts'') 
in view oil JS 6,704,048 ("Malkin") - section & of the final office action. 

Claims 9 and 25 stand rejected under 35 I JSC 103(a) per the other claims and further in 
view of US 5,778,082 ("Chu") - section 9 of the final office action. 

There arc no other claims. 

All rejected claims arc being appealed. 

IV. STATUS OP AMENDMENTS 

There was a first communication under rule 116. In a first advisory action, the Examiner 
said that this first communication overcame all rejections over Baker In a second advisory 
action, correcting the first advisory action, the Examiner said that the first communication under 
rule 1 1 6 could not be entered due to an error in the claim identifier for one of the claims. 

Tn a telephone conversation dated October 1 9 ? the Examiner indicated that if Applicants 
corrected the claim identifier, all rejections other than paragraphs 8 and 9 of the office action 
would be overcome, 

A second communication under rule 1 16 was faxed in on October 20, 2004, correcting 
the problem with the claim identifier, repeating the amendments to the claims, and incorporating 
the prior arguments by reference. Applicants therefore believe that the only parts of the office 
action chat remain to be overcome arc paragraphs 8 and 9, though Applicants have not yet 
received a third advisory action stating this in writing. 

Accordingly, in reliance on the Examiner's statements during the telephone interview, 
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Applicants will only argue against sections 8 & 9 of the office action. 

V. SUMMARY OF TIIE CLAIMED SUBJECT MATTER 
Claim 1 

The claimed invention relates to a video conferencing system. The system includes a 
stationary image pickup device (ref #210; spec p. 4, 11. 12, 1 8 ? 20; p. 5, 11. 3-10 & 19; p. 6, 1. 10). 
The image pickup device remains motionless during operation (spec. p. 5, 11, 4, 8, 9, 19). The 
system also includes an audio pickup device (ref. #'s 231, 232; spec. p. 4 7 1. 13-14; p. 5, 1. 19) for 
generating audio signals (ref, # 235; spec. p. 5, 1. 20) representative of sound from an audio 
source. The system also includes means (ref # 270; spec. p. 5, 1. 20 through p. 6, 1. 9) for 
processing the image signals and audio signals to determine a direction of the audio source 
relative to a reference point The determination of direction depends at least at times on the 
image signals. 

Claim 5 

This claim recites the video conferencing system of claim 1, further comprising an 
electronic pan tilt zoom system for electronically manipulating the image signals to effectively 
provide at least one of variable pan, tilt, and /.oom functions, (spec. p. 6, 11. 1 0-1 5) 

Claim 2 

This claim recites the video conferencing system of claim 1, wherein said processing 
means (ref. #270; spec. p. 5, L 20 through p. 6, 1. 9) comprises: 
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an audio source localization system (ref. #240, spec, p, 5, 1. 21); 

a computer vision person detection system (rcf #250, spec. p. 5. 1. 22); and 

a multimodal speaker detection system (ref. #260, spec, p. 5 ? 1. 22 through p, 6, 1. 9). 

Claim 3 

This claim recites the video conferencing system of claim 2 ? further comprising an 
integrated housing (rcf. #110, spec. p. 6, 1. 14 ct. scq.) for an integrated video conferencing 
system incorporating the image pickup device (ref #210; spec p. 4, 11. 12, 18, 20; p. 5, 11. 3-10 & 
1 9; p. 6, 1. 1 0), the audio pickup device (ref. #s 23 1 , 232; spec. p. 4, 1. 1 3- 1 4; p. 5,1.1 9), and the 
processing means (ref. #270; spec. p. 5, L 20 through p. 6, 1. 9). 

Claim 4 

This claim recites the video conferencing system of claim 3, wherein the integrated 
housing (rcf. #1 10, spec. p. 6 J. 1 4 el. seq.) is sized for being portable. The portability is a 
functional advantage. 

Claim 6 

This claim recites the video conferencing system of claim 1, wherein the image pickup 
device (rcf #210; spec p. 4, 11. 12, 18, 20; p. 5, 11. 3-10 & 19; p. 6, 1. 10) is a stationary camera 
that remains motionless during operation of the video conferencing system. 
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Claim 8 

™* data recta ,„ e vlto> ronfercnci „ 8 ^ of cWm 2 aud . o ^ 

locals system (re f W4 „. spec, p. 5, ,. 2 „ ^ m _ of 1Ilc ^ ^ ^ 

fa audio sou™ moves rcWve to d- refere „ce point, .,4 „, risplmse t0 to raCTOt ^ 
-*> « .o^on syslem caases a chmgc , n a M rf v . cw ^.^ ^ pMup ^ 

(spec P- 5, n. 8-9). „ ahouid te nutctl ta cIaim sU]1 depmds fom n ( wMch s(ate 

that the image pickup device is motionless. 

Claim i n 

l«s claim has recitations analogous to those of claim 1 and further recite* manipulating 
the image signals u, produce refined image signal, (spec, p. 3, II. 4-9) dependmg on the 
determined direction; and oulputting the rotted i ma ge ., igntl l 3 . F^mpics of refined image 
signals arc discussed in the specification further with respect to simulating pan/tih/zoom 
functions as discussed with respect to claim 5. 


Claim 1 1 

This claim depends from claim 1 0 and contains recitations analogous to those of claims 2 


and 5. 
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Claim 1 2 

This claim depends from claim 1 0 and recites thai manipulating the image signals 
includes varying a field of view of the image pickup device in response to the control signals, 
'Hie spec uses the pan/ti It/zoom feature as an example of varying the field of view, as explained 
with respect to claim 5. 

Claim 14 

This claim depends from claim 1 0, but contains recitations similar to those of claim 8. 
The reader is referred to the summary of claim 8, above. 

Claim 15 

This claim recites the method of claim 13, wherein processing the image signals includes 
generating control signals (265, spec. p. 6, 11. 1-2) depending on the audio based direction, and 
manipulating the image includes electronically panning, tilting, and/or zooming said image 
pickup device depending on the control signals (spec, p, 6, II. 10-15). 

Claim 16 

This claim recites a video conferencing system comprising: 

microphones for generating audio signals representative of sound from a 

speaker; 

a stationary video camera, remaining motionless during operation, for 
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generating video signals representative of a video image; 

an electronic pan tilt zoom system for manipulating video images to 
produce the visual effects of panning, tilting, and/or zooming; 

a processor for processing the video signals and the audio signals u> 
determine a direction of a speaker relative to a reference point and supplying 
control signals to the electronic pan tilt zoom system lor producing images that 
include the speaker in the field of view of the camera, the determination of 
direction depending at least at times on the video signals, the control signals being 
generated based on the determined direction of the speaker; and 

a transmitter for transmitting audio and video signals for video 
conlerencing. 

Accordingly, This claim includes, inter alia, the stationary/motionless limitation discussed 
with respect to claim 1 and the pan/tilt/zoom system limitation discussed with respect to claim 
5 1 . 

Claim 17 

This claim recites the video conferencing system of claim 1, wherein at limes the 
determination of the direction of the audio source depends on both the image signals and the 
audio signals (spec. p. 5 ? line 1 8 through p. 6, line 9). 


1 The claim is different in scope from claim 5, however, because it does not contain the limitation 
from claim 1 that the determination of direction depends at least at times on the image signals. 
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Claim 1 8 

This claim recites the video conferencing system of claim 1, wherein Hie processing 
includes determining the movement of the audio source depending til least at times on the image 
signals, (spec. p. 3, 1. 7; p. 5, 1. 18 through p. 6, 1. 9). 


Cl aim 19 

This claim recites The video conferencing system of claim 1 , wherein (he processing 
includes tracking the position of the audio source when the audio source moves, the tracking 
depending at least at times on the image signals (spec. p. 3, 1. 7; p. 5, 1. 1 8 through p. 6, 1. 9). 

Claim 20 

This claim recites the video conferencing system of claim 2, wherein the computer vision 
person detection system detects the movement of the audio source when the audio source moves 
relative to the reference point, and, in response to the movement, the computer vision person 
detection system causes a change in a field of view of the image pickup device, (spec. p. 3, 1. 7; 
p. 5,1. 18 through p. 6, 1. 9) 

Claim 21 

This claim recites the method of claim 10, wherein processing the image signals further 
includes: 

detecting the movement of the audio source when the audio source moves; 

and 
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causing electronically, in response to the movement, an [sic] variation in a field of 
view of the image pickup device, (spec. p. 3, L 7; p. 5, 1. 18 through p. 6, 1. 9) 

Claim 22 

The recitations of this claim are similar to those of claim 1 8, except that the claim 
depends from claim 10, rather than claim 1 . 

Claim 23 

The recitations of" this claim are similar to those of claim 1 9, except that the claim 
depends from claim 1 0, rather than claim 1 . 

.Claim_24 

lliis claim recites a video conferencing system, hut is otherwise analogous to claim 10. 

VI. GROUNDS Ol' REJECTION TO BE REVIEWED ON APPEAL 

Section 8 of the find office action is to be reviewed, per the telephone conference of 

Octohcr 19, wherein the Examiner indicated thai the prior sections would be overcome hy the 

amendment of October 20. 

Section 9 of the office action would appear to be moot. 
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VIL TtiF. ARGUMENT 

Section 8 of the Fiual Office Action 

Section J? of the Office Action purports to reject 23 clai™ over a combination of 
references. The references are long and complex. The Polls document contain* 70 pages and 
25 sheets of figures. The MaJkin document contains 12 columns of text and 8 sheets of 
drawing. 7 Therefore, in keeping with 37 C.RR. 1 .104 (c) (ii) the Examiner is required to specify 
what part of the references is relied upon in rejecting each claim. 

Instead, the Examiner has grouped all of the rejections together, wiihout indicating which 
element of which claim is rejected over which part of the references. The comments made are 
noL clearly applicable to all claims. Accordingly, Applicants respectfully submit that the 
rejections are improper. 


The Potts reference: clai m X 

Applicants respectfully submit that paragraph 8 of the final office action fails to make a 
prima facie case of obviousness against claim 1. 

The Examiner mischaracterizes Applicants amendment of February, 2004, by stating that 
page 7, second & third paragraphs, makes admissions as to the Potts reference. This is false. 
Those paragraphs of the amendment do not refer to the Potts reference at all. Iliose paragraphs 
of the amendment refer to the summary section of Applicants' own specification at pages 2 and 
3, and iherefore do not characterize any prior art. 


The section also cites the Baker reference: however, since the Examiner has indicted that the R«kcr reft 
withdrawn, Applicants will only discuss the Potts & Mnlkin references. 
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The WO 99/60788 document ("Potts"), which is referred to in the fourth paragraph, is 
only cited by the amendment for tracking the image of a moving speaker - namely for a camera 
in motion - not for a device that is motionless. 

Accordingly, Applicants have not made the admissions as cfiaracteriy.ed by the Examiner 
in the Final Office Action. 

Moreover, the Examiner misconstrues Applicants' claims. The Examiner alleges that the 
claims recite "the camera nut being motionless during operation" [emphasis added]. In fact, 
Claim 1 recites that the image pickup device is motionless during operation. Accordingly, the 
Examiner's statements about pan, tilt, zoom cameras are completely irrelevant to Applicants' 
claim. Since Malkin is apparently cited only for this feature, Mallcin, at least as applied, would 
also appear to be irrelevant to the claim. 

Tn short, the Examiner wholly fails to read Lhe elements of claim 1 on the references; and 
makes false and irrelevant statements about Applicants' claims and Applicants' amendment in 
support of the rejection. 

The Board is respectfully requested to remand this application to the Examiner for 
issuance of a rejection that complies wilb 37 CFR 1 . 1 04. 

Claim 5 

The office action makes no explicit rejection of this claim as distinguished from the other 
claims; however, it appears that the Examiner misconstrues this claim. He apparently reads it as 
requiring a pan, tilt and zoom camera . That is not what is recited. What is recited is a system 
for electronically manipulating the image signals. It is this manipulation that provides pan, till, 
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and/or 700m functions . The image pickup is still the same motionless device that was cited in 
claim I. 

Accordingly, as mentioned above, the references cited for pan, tilt, zoom cameras arc 
irrelevant. The Examiner has Tailed to indicate any reference that teaches or suggests pan, tilt, 
zoom functions being generating in a motionless image pickup device. Applicants accordingly 
respectfully submit Lhal the Examiner has failed to meet his burden of making a prima facie case 
against this claim. 

Claim 2 

The Examiner fails to indicate where the references teach or suggest these features. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a. prima facie case against this claim. 

Claim 3 

The Examiner fails to indicate where the references teach or suggest these features. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 

Claim 4 

The Examiner fails to indicate where the references teach or suggest this feature or die 
resulting functional advantage. Applicants accordingly respectfully submit that the Examiner has 
failed to meet his burden of making a prima facie case against this claim. 
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Claim 6 

The Examiner has failed to indicate any reference thai teaches or suggests this feature. 
Applicants accordingly respectfully submit that the lixamincr has failed to meet his burden of 
making a prima facie case against this claim. 

Claim 8 

The lixamincr fails to indicate where the references teach or suggest the recited features 
or the resulting functional advantage. Applicants accordingly respectfully submit that the 
Examiner has failed to meet his burden of making a prima facie case against this claim. 

Claims I0&24 

The refined image signals are a functional advantage. Accordingly;, the claim 
distinguishes even more clearly over the references than claim 1 does. 

The Examiner has completely Tailed to indicate where these recitations or the 
accompanying functional advantage are Uiught or suggested in the references. Applicants 
accordingly respectfully submit that the Examiner has failed to meet his burden of making a 
prima facie case against this claim. 

Claim 24 is analogous. 
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Claim..H 

This claim depends Irom claim 10 and contains recitations analogous to those of claims 2 
and 5. Accordingly the arguments applicable to those claims apply to this claim. 

Claim 1 2 

The lixaminer fails to indicate where the references teach or suggest this feature or the 
functional advantage of varying a field of view. Applicants accordingly respectfully submit that 
the Examiner has Tailed to meet his burden of making a prima facie case against this claim. 

Cla im 14 

This claim depends from claim 10, but contains recitations similar to those of claim 8. 
Arguments? applicable to claim 8 are therefore applicable to claim 14. 

Claim 15 

The Examiner fails to indicate where the references teach or suggest this feature. 
Applicants accordingly respectfully submit that the Kxamincr has railed to meet his burden of 
making a prima facie case against this claim. 

Claim 16 

This claim includes, inter alia, the stationary/motionless limitation discussed with respect 
to claim 1 and the pan/tilt/zoom system limitation discussed with respect to claim 5. The 
rejection is accordingly deficient with respect to these limitations, as discussed before. 
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Claim 17 

The Examiner I ails to indicate, where the references leach or suggest this feature. 
Applicants accordingly respectfully submit that die Examiner has failed to meet his burden of 
making a prima facie case against this claim. 

Claim IS 

The Examiner fails to indicate where the references teach or suggests this feature. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 

Claim 19 

The Examiner fails to indicate where the references teach or suggests this featured 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden or 
making a prima facie case against this claim. 

Claim 20 

The Examiner fails to indicate where the references teach or suggests these features. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making * prima facie case against this claim. The functional advantage of changing field of view 
has also not been read on the references in paragraph 8 or 9. 
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The Examiner fails to indicate where the references teach or suggests these features. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a. prima facie case against this claim. 

Cl aim 22 

The Kxamincr fails to indicate where the references Teach or suggests this feature. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against tin's claim. 

Claim 23 

The Examiner fails to indicate where the references leach or suggests this feature. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making & prima facie case against this claim. 
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p. 20 


Vlli. CONCLUSION 

Applicants respectfully submit that they have answered each issue raised by the Examiner 
and that the application is accordingly in condition for allowance. Such allowance is therefore 
respectfully requested. 

Respectfully submitted, 


Anne E. Barschall 
Reg. No. 31,089 
(914)332-1019 
fax 914-332-7719 
February 14, 2006 
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CLAIM APPENDIX 


, 1 . (previously presented) A video conferencing system comprising: 

a stationary image pickup device, remaining motionless during operation, for generating 

3 image signals representative of an image; 

an audio pickup device lor generating audio signals reprcacnUUivc of sound from an audio 

5 source; and 

6 means for processing said image signals and said audio signals to determine a direction 

7 of the audio source relative to a reference point, the determination of direction depending at least 

8 at times on the image signals. 

, 2. (previously presented) The video conferencing system of claim 1 wherein said processing 

2 means comprises: 

3 an audio source localization system; 

4 a computer vision, person detection system; and 
s a multimodal speaker detection system. 


3. (previously presented) The video conferencing system of claim 2, farther comprising an 
integrated housing for an integrated video conferencing system incorporating the image pickup 
device, the audio pickup device, and the processing means. 
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4. (original) The video conferencing system of claim 3, wherein the integrated housing is sized 
for being portable. 

1 5, (previously presented) The video conferencing system ol" claim 1, further comprising an 

2 electronic pan tilt zoom system for electronically manipulating the image signals to effectively 

3 provide at least one of variable pan, tilt, and y.oom functions. 

1 6. (previously presented) The video conferencing system of claim 1 , wherein the image pickup 

2 device is a stationary camera thai remains motionless during operation of the video conferencing 

3 system, 

7. (previously presented) The video conferencing system of claim I , wherein the processing 
means provides control signals to an electronic pan tilt zoom system. 


8, (previously presented) The video conferencing system of claim 2, wherein the audio source 
localization system detects the movement of the audio source when the audio source moves 
relative to the reference point, and, in response to the movement, the audio source localization 
system causes a change in a field of view of the image pickup device, 

9. (previously presented) The video conferencing system of claim I , wherein the audio pickup 
device is comprised of an array of two microphones. 
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i 1 0. (previously presented) A method comprising the steps of: 

* generating, at a stationary image pickup device, remaining motionless during operation, 

3 image signals representative of an image; 

4 generating, at an audio pickup device, audio signals representative of sound from an 
s audio source; 

6 processing the image signals and the audio signals to determine a direction of the audio 

7 source relative to a reference point, the determination of direction depending at least at times on 

8 the image signals; 

9 manipulating the image signals to produce refined image signals depending on the 

10 determined direction; and 

11 outputting said refined image signals. 


i 11. (previously presented) The method of claim 10 further comprising the steps of: 

i applying said audio signals to an audio source localisation system; 

:i applying said image signals to a computer vision person detection system; 

i processing said audio signals and said image signals with a multimodal speaker detection 

5 system to determine the direction of the audio source; 

a generating control signals based on the determined direction of the audio source; 

7 applying the control signals to an electronic pan till zoom system to mimic the effect of at 

8 least one function of a movable camera, said function selected from the group consisting 
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v panning, tilling, and zooming said movable camera; and 
u> providing an output from said electronic pan tilt zoom system. 

12. (previously presented) The method of claim 1 0, wherein manipulating the image signals 
includes varying a field of view of the image pickup device in response to the control signals. 

1 3. (original) The method of claim 10, wherein processing the audio signals includes determining 
an audio based direction of the audio source based on Lhe audio signals. 

1 1 4. (previously presented) The method of claim 1 0, wherein processing the audio signals 

2 includes detecting the movement of the audio source when ihc audio source moves; and 

j manipulating the image signals includes causing electronically, in response to the 

4 movement, a variation in a field of view of the image pickup device. 

1 1 5. (previously presented) The method of claim 1 3, wherein processing lhe image signals 

2 includes generating control signals depending on the audio based direction, and manipulating the 

3 image includes electronically panning, tilting, and/or zooming said image pickup device 

4 depending on the control signals. 

1 1 6. (previously presented) A video conferencing system comprising: 

2 microphones for generating audio signals representative of sound from a speaker; 
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x a stationary video camera, remaining motionless during operation, for generating video 

•i signal* representative of a video image; 

5 an electronic pan tilt zoom system for manipulating video images to produce the visual 

6 effects of panning, lilting, and/or zooming; 

7 a processor for processing the video signals and the audio signals to determine a direction 
x of a speaker relative to a reference point and supplying control signals to the electronic pan tilt 

• zoom system for producing images that include the speaker in the field of view of the camera, the 
determination of direction depending at least at times on the video signals, the control signals 
being generated based on the determined direction of the speaker; and 

a transmitter for transmitting audio and video signals for video conferencing. 


10 


12 


1 1 7. (previously presented) The video conferencing system of claim 1 , wherein at times the 

2 determination of the direction of the audio source depends on both the image signals and the 

3 audio signals, 

, 18. (previously presented) The video conferencing system or claim I , wherein the processing 

2 includes determining the movement of the audio source depending at icasL at time, on the image 

3 signals. 

, 19. (previously printed) The video conferencing system ordain. I, wherein the processing 
2 includes tracking the position of the audio source when the audio source moves, the tracking 
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3 depending at least at times on the image signals. 

1 20. (previously presented) The video conferencing system of claim 2, wherein the computer 

2 vision person detection system detects the movement of the audio source when the audio source 

3 moves relative to the reference point, and, in response to the movement, the computer vision 

4 person detection system causes a change in a field of view of the imago pickup device. 

1 21. (previously presented) The method of claim 1 0, wherein processing the image signals further 

2 includes: 

a detecting the movement of the audio source when the audio source moves.; and 

< causing electronically, in response to the movement, an variation in a field of view of the 

5 image pickup device. 

22, (previously presented) The method of claim 10, wherein the processing includes determining 
the movement of the audio anurcc depending ul least at limes on the image signals. 
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23. (previously presented) The method of claim 10, wherein the processing includes tracking the 
position of the audio source when the audio source moves, the tracking depending at least at 
times on the image signals. 

i 24. (previously presented) A video conferencing system, comprising: 

i a stationary image pickup device, remaining motionless during operation, for generating 

3 image signals representative of an image; 

4 an audio pickup device for generating audio signals representative of sound from an audio 

5 source" 

meant) for processing the image signals and the audio signals to determine a direction of 

7 the audio source relative to a reference point, the determination of direction depending at least at 

g times on the image signals; 

9 means for manipulating the image signals to produce refined image signals depending on 

10 the determined direction; and 

it an output for outputting said refined image signals. 

25. (previously presented) The video conferencing system of claim 9, wherein the array of 
microphones incJudcs only two microphones. 
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EVIDENCE APPENDIX 
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RELATED PROCEEDINGS APPENDIX 
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